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Unstated  Needs  are  Elusive  but  Important 


“It's  really  hard  to  design  products  by  focus  groups.  A  lot  of  times, 
people  don't  know  what  they  want  until  you  show  it  to  them ” 

(Steve  Jobs) 

“If  I  had  asked  people  what  they  wanted,  they  would  have  said  faster 
horses”  (Apocryphal,  attributed  to  Henry  Ford) 


When  needs  go  unrecognized,  critical  features  get  overlooked,  including: 

•  Non-functional  requirements  that  drive  architecture 

•  Innovative  product  and  service  features  that  sustain  customer  loyalty 

And  if  left  undiscovered,  can  result  in: 

•  Stakeholder/user  disruption  and  frustration 

•  Requirements  volatility,  expensive  rework,  delays 
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Existing  Requirements  Elicitation  Methods 

Limitations  in  what  requirements  specifications  convey: 

Needs  that  are  more  likely  to  be  salient  to  the  stakeholder/user 
-  Don't  know  what  they  don't  know 


v. 


Motivation  for  requirements  is  often  lost  or  missing 


Other  limitations  with  existing  methods  include: 


J 


Timeboxed,  workshop-style  meetings 
that  can’t  be  held  virtually 

Missing  stakeholders 

Won't  work  at  scale 


r 


EURS  aims  to  develop  and  validate  a  scalable  method  for  determining  the  unstated  needs 
of  stakeholders,  which  result  in  a  more  innovative  set  of  requirements  as  the  basis  for 
subsequent  system  design,  implementation,  deployment,  sustainment,  and  modernization. 
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Overview  of  SEI  KJ+  Approach 


Start  with  known 
requirements 

(obtained  by  existing 
elicitation  method) 


Step  1 :  Design 
open-ended, 
probing  questions 
to  interview  users  on 
extreme  positive 
and  negative 
experiences 


Step  2:  Conduct 
interviews  focusing  on 
context 
not  solutions 
(asking  “how”  and  ’’why”) 


Step  3:  Mine 
interview  output  and 

characterize 
experiences  in 

terms  of  action, 
motivation,  context) 


Innovative 
Requirements 


Step  6:  Triage  need- 
solution  pairs  into: 

•  Delighters 

•  Satisfiers 

•  Must-be’s 
(Kano  analysis) 


Step  5:  Derive 

unstated  needs  and 
brainstorm  candidate 
innovative  solutions 


Step  4:  Look  across 
characterizations  to 
Identify  themes  of 

experience 

(KJ+  Affinitization) 
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VIRTUALIZE - >< -  AUTOMATE 


EURS  Research  Focus 


t  Summarized 
meta 

discussion 
data 


Textual 

processing 


KJ+  process 

redesigned 
and  tool- 

enabled  to 

facilitate 
offline  and 
virtual 

performance 


“No  frills” 
Ideation 


Facilitated 
telecons 
(take  turns) 
supplemented 
by  MS  Word, 
Excel  to  manage 
data 


Traditional  KJ 

Face  to  Face 

conducted  in  a 

workshop 

V 

setting 

Need  to 
synchronize 
only  in  step  2 
(interviews) 

Other  steps  are 
asynchronous 
but  facilitated 


Discussion 
with  real  time 
data  feedback 


Ideation  with 
structured 
facilitation 


Asynchronous 
discussion 
with  data 
feedback 


Ideation  fully 
Asynchronous 
and  continuous 


■ocus 
of  FY1 4 
research 
Questions. 


5-10  People 

SCALE 


11-25  People 


26-100  People 


100+  People 


FY14  Task  structure 


1 .  Redesign  the  KJ  method  for  virtual,  asynchronous  use  to  the 
maximum  extent  possible  with  minimal  technology,  creating  KJ+ 

2.  Develop  pilot  training,  process  scripts,  briefing  templates,  and 
tooling  to  support  pilot  of  KJ+ 

3.  Select  pilot  candidate  and  train  team 

4.  Conduct  a  small-to-moderate  scale  A-B  comparison  experiment 

a)  Project  requirements  are  developed  without  KJ+ 

b)  KJ+  is  then  applied 

c)  New  requirements  are  identified  and  effort  is  recorded 

5.  Analyze  results 
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FY14  Accomplishments 


Key  SEI  Activities/Contributions 

•  Adapted  existing  workshop-based  method  for  distributed  use 

•  Conducted  A-B  Comparison:  pilot  was  successful 

•  Pilot  collaborator  intends  to  adopt  the  KJ+  method  and  has  funded  our 
continued  engagement 

•  Will  pursue  publication  in  Requirements  Engineering  Journal 

Revisiting  our  Research  Questions:  Can  KJ  be  adapted  for: 

•  Virtual,  small-to-moderate  scale  use? 

•  Identifying  innovative  requirements  that  anticipate  unstated  needs? 

•  Increasing  end-user  satisfaction? 

•  Mitigating  requirements  volatility? 

•  Reducing  sustainment  and  modernization  costs? 
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Next  Steps 


Ultra-scale  experiment  allowing  much  more  participation  in 
developing  requirements: 

•  Increase  #  of  participants  by  introducing  ideation 

•  Affinitize  visually  as  well  as  virtually 

•  Automate  identification  of  relevant  ideas  to  keep  others  informed  of 
what  might  interest  them 

-through  machine  learning  and  probabilistic  topic  modeling 


Contact  us  if  you  would  like  to  collaborate. 
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Contact  Information 

Presenter  /  Point  of  Contact 

Mike  Konrad 

Software  Solutions  Division 
Telephone:  +1  412-268-5813 
Email:  mdk@sei.cmu.edu 


Web 

www.sei.cmu.edu 

www.sei.cmu.edu/contact.cfm 


U.S.  Mail 

Software  Engineering  Institute 
Customer  Relations 
4500  Fifth  Avenue 
Pittsburgh,  PA  15213-2612 
USA 

Customer  Relations 

Email:  info@sei.cmu.edu 
Telephone:  +1  412-268-5800 

SEI  Phone:  +1  412-268-5800 

SEI  Fax:  +1  412-268-6257 
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